Conventional neural network architectures may feature globally interconnected layers of processing components. A processing component of one layer of the neural network architecture is connected to all the processing components in another layer. This can lead to concerns about wiring. For example, as the number of processing components increases linearly, the number of wires required to maintain global interconnection increases quadratically. Furthermore, the length of wires required to maintain global interconnection increases linearly. Thus, the amount of wire increases cubically. This limits conventional global interconnection systems.
Conventional neural network architectures may also be organized in a manner where approximating functions are expressed as a sum of products. The sum of products organization can lead to a rapid expansion of terms in complex problems.
Conventional neural network training can involve significant off-line, serial processing of values, weights, thresholds and other computational data. Typically, neural network training involves a global simulation of neuronal behavior and adjustment of weights on connections between processing components, inputs, and/or outputs. The typical training employs off line approximations that grow rapidly in processing time as the complexity of the neural network (e.g., number of neurons, number of layers) increases. When substantially all the training calculations are performed in serial by a single processor or a small set of cooperating processors, training large, complex neural networks becomes computationally infeasible.
In traditional neural network architectures, substantially all the processing components perform a similar transformation function on a fixed set of inputs. While the function may be similar, the processing component may be assigned a local threshold value for the transformation function. Again, central calculation and recalculation of the threshold value can be computationally cumbersome. Furthermore, having substantially all the processing components perform a similar transformation function may make conventional architectures less flexible than is possible.